NUMBER OF VERTICES IN GELFAND-ZETLIN POLYTOPES 
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Abstract. We discuss the problem of counting vertices in Gelfand-Zetlin poly- 
topes. Namely, we deduce a partial differential equation with constant coefficients 
on the exponential generating function for these numbers. For some particular 
classes of Gelfand-Zetlin polytopes, the number of vertices can be given by ex- 
plicit formulas. 



1. Introduction and statement of results 

Gelfand-Zetlin polytopes play an important role in representation theory \GZ\ 10] 
and in algebraic geometry (see |KST] ). Let Ai ^ ... ^ A„ be a non-decreasing 
finite sequence of integers, i.e. an integer partition. The corresponding Gelfand- 

rt(n — 1) 

Zetlin polytope is a convex polytope in R 2 defined by an explicit set of linear 
inequalities depending on Aj. It will be convenient to label the coordinates Uij in 

IR ( 2 ' by pairs of integers (i, j), where i runs from 1 to n — 1, and j runs from 1 
to n — i. The inequalities defining the Gelfand-Zetlin polytope can be visualized by 
the following triangular table. 

Ai A2 A3 ... A n 

UlA Ul2 • • • «l,n-l 



U 2 ,l ■ ■ ■ V>2,n-2 

Un-2,1 Un~2,2 
«n-l,l 



(GZ) 



where every triple of numbers a, b, c that appear in the table as vertices of the 
triangle 

a b 

c 

are subject to the inequalities a ^ c ^ b. 

In this paper, we discuss generating functions for the number of vertices in 
Gelfand-Zetlin polytopes. We will use the multiplicative notation for partitions, 
e.g. l Jl 2* 2 3* 3 will denote the partition consisting of i\ copies of 1, i 2 copies of 2, and 
«3 copies of 3. Given a partition p, we write GZ(p) for the corresponding Gelfand- 
Zetlin polytope, and V(p) for the number of vertices in GZ(p). 

Fix a positive integer k, and consider all partitions of the form l n . . . k lk , where 
a priori some of the powers ij may be zero. We let denote the exponential 
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generating function for the numbers V(l 11 . . . k lk ), i.e. the formal power series 

E k = V V(l l \..k 1 *)^...^. 

Our first result is a partial differential equation on the function E k : 

Theorem 1.1. The formal power series E k satisfies the following partial differential 
equation with constant coefficients: 

d k ( d d \ f d d \ \ „ 



dzi...dz k \dz\ dz 2 J \dz k -i dz k 
E.g. we have 

E 1 (z 1 )=e z \ E 2 (z u z 2 )=e z ^I (2^- 2 ), 

where Iq is the modified Bessel function of the first kind with parameter 0. This 
function can be defined e.g. by its power expansion 

n=0 

It is also useful to consider ordinary generating functions for the numbers V^l* 1 . . . k lk ] 
G k {y u ...,y k )= V{l il ...h*')y£...yZ. 

ii,...,ifc>0 

We will also deduce equations on G k . These will be difference equations rather than 
differential equations. For any power series / in the variables yx, . . . , y k , define the 
action of the divided difference operator A, ; on / as 

A . (/) = f-f\y,=0 _ 

Hi 

Theorem 1.2. The ordinary generating function G k satisfies the following equation 
{A 1 ...A k - (Ai + A 2 ) . . . (A fc _x + A fc )) G k = 0. 

It is known that the ordinary generating functions G k can be obtained from expo- 
nential generating functions E k by the Laplace transform. Thus Theorem 1 1 . 2 1 can in 
principle be deduced from Theorem 1 1 . 1 1 and the properties of the Laplace transform. 
However, we will give a direct proof. 

For k = 1, 2 and 3, the generating functions G k can be computed explicitly. It is 
easy to see that 

Gi{yi) = — — , G 2 (2/1,1/2) 



1-3/1-2/2 
We will prove the following theorem: 
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Theorem 1.3. The function G 3 (x,y,z) is equal to 

2xz — y(l — x — z) — y^l — 2{x + z) + (x — z) 2 
2(1 - x - z)((x + y)(y + z) - y) ' 

The numbers Vk t e,m — V(l k 2 e 3 m ) can be alternatively expressed as coefficients of 
certain polynomials: 

Theorem 1.4. The number V^ m for k > 0, £ > 0, m > is equal to the coefficient 
with x k z m in the polynomial 

1 — XZ xfc+£+m/-| _|_ \k+i+m / , „\k+t+m\ 



1 + XZ 



((1 + x) k+t+m (l + Z ) k +"+ m - ( X + z f+*+™} 



Set s = k + £ + m. Note that, since the term (x + z) s is homogeneous of degree 
s, the number Vk^m, where k,£,m> 0, is also equal to the coefficient with x k z m in 
the power series 

(l-xz)(l + x) s (l + z) s 

1 + XZ 

This implies the following explicit formula for the numbers Vk/ <m (k,£,m > 0): 

k 



(:)(:)+ 2 b- 1 ^ 



s 

k — i I \m — i 



Note that the sum Yli=i(~ 1)* (felj im-i) can ^ e expressed as the value of the gen- 
eralized hypergeometric function 3 F 2 , namely, it is equal to { k t^) ( m li) 3^2(1,1 — 
k,l - m;2 + £ + m,2 + k + £; -1). 



2. Recurrence relations 

Let i? be the polynomial ring in countably many variables x±, X2, £3, .... Define 
a linear operator A : R — > R by the following formula: 

A{Xi 1 Xi 2 ■ ■ ■ Xi k f^j ~\~ %i2 ) (%i2 Xi 3 ) • • • (Xik—i )/• 

In this formula, x^, . . . , Xi k is any finite subset of variables, and f(xi 1 ,Xi 2 , . . . , Xi k ) 
is any polynomial in these variables. The formula displayed above defines a linear 
action of A on the entire R. Indeed, any polynomial P G R can be represented 
as the sum P = J2 S P s , where S runs through all finite subsets of variables, and 
Ps denotes the sum of all terms (monomials together with coefficients) that involve 
exactly all variables from S and no other variables. The polynomial A(Pg) is defined 
above, and we extend A by linearity. We also set A(l) = 1 by definition. 

The operator A thus defined reduces the degrees of all nonconstant polynomials. 
Therefore, for any polynomial P, there exists a positive integer N such that A N (P) 
is a constant, which is independent of the choice of N provided that N is sufficiently 
large. We let A°°(P) denote this constant. 
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Proposition 2.1. We have 

V{l n ...k lk ) = A°°(x? ...xi). 

Proof. We will argue by induction on the degree ii + • • ■ + ik, equivalently, on the 
dimension of the Gelfand-Zetlin polytope GZ(1 11 . . . k lk ). Let 7r be the linear pro- 
jection of GZ{1 %1 . . . k lk ) to the cube C given in coordinates (ui, . . . ,Uk-i) by the 
inequalities 

l^ Ul ^2^u 2 ^...^k-l^ ^ k. (C) 

Namely, we set u x = u ljh , u 2 = u 1M+h , . . . , u k _ x = u hh+ ... +ik _ 1 . Observe that all 
vertices of GZ(p) project to vertices of the cube C. Thus it suffices to describe the 
fibers of the projection 7r over the vertices of the cube C. 

It will be convenient to label the vertices of the cube C by the monomials in the 
expansion of the polynomial A(x\ . . .x^). Namely, to fix a vertex of C, one needs 
to specify, for every j between 1 and k — 1, which of the two inequalities j ^ Uj 
or Uj ^ j + 1 turns to an equality. Similarly, to fix a monomial in the polynomial 
A(xi . . . Xk), one needs to specify, for every j between 1 and k — 1, which term is 
taken from the factor (xj + Xj+i), the term Xj or the term Xj + \. This description 
makes the correspondence clear. 

Let v be the vertex of the cube C corresponding to a monomial x^ 1 . . . x^ k . It is 
not hard to see that the polytope 7r _1 (t>) is combinatorially equivalent to 

Suppose that 

A{x\ . . . Xk) = ^ ^ C ai ___ ak X^ . . . Xfc . 
ai,...,a k 

Then we have 

V(1 H ...k lk ) = C ai...a k V(l h ~ 1+ai . . . k ik ~ 1+ak ). 

a lt ...,a k 

Since for any /c-tuple of indices cti, . . . , a^, for which the corresponding coeffi- 
cient c aij ... jQfc is nonzero, the Gelfand-Zetlin polytope GZ(l ll ~ 1+OL1 . . . k lk ~ l+ak ) has 
smaller dimension than GZ(1 %1 . . . k %k ), we can assume by induction that 

y^ii-l+Qi foik-l+c>k^ _ ^ii-l+ai X k k ~ 1+ak ) 

Hence we have 

V(l h ...k ik )= Cai-a* ^°°(^r 1+Ql • • • 4 fc " 1+Qfc ) = A°°(A(x\' ...X ik )). 

ai,...,a k 



The desired statement follows. 



□ 
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3. Equations on generating functions E k and G k 

In this section, we deduce equations on the generating functions E k and G k . In 
particular, we prove Theorems 11.11 and 11.21 

For a multi-index a = (aci, . . . , a k ), we let z a denote the monomial z® 1 . . . z^ k , and 
a\ denote the product a.\\ . . . a k \. The partial derivation with respect to zi will be 
written as di. The power d a will mean d" 1 . . . d^ k . We will write If for the operator 
of integration with respect to the variable Z£. This operator acts on the power series 
Y^n=o a n z e-> where a n are power series in the other variables, as follows: 




x z e+1 

n=0 



We will use the expansion 

(xi + x 2 ) ... (x k -i + x k ) = 22 Ca%C 



a 



in which the coefficients be computed explicitly. Let E% be the sum of all 

terms in E k divisible by z\...z k . Then we have (i, j, a being multi- indices of 
dimension k) 



v ^ v ^ 'oof„i-l+a\ Z 



j>0 i>0 a 



X 



= y: catrh ...i k j2 A ^- 1+a ) ( Si7 a v. = £ Cadah ■ • • J * ?! 

Apply the differential operator d± . . . d k to both sides of this equation. Note that 
dx . . . d k {E* k ) = di... d k {E k ). Thus we have 

d x . . . d k (E k ) = c a d a A 00 ^)* - 
Observe also that, since a ^ whenever c a ^ 0, we have 

Z3 



This implies Theorem 11.11 

Example: k = 1 and k = 2. In the case k = 1, we have E\ = e zi . Consider now 
the case k = 2. Set E = E 2 , x = Z\ and y = z 2 - By Theorem 11.11 the function E 
satisfies the following partial differential equation: 

-^xy E x ~\- Ey. 

This equation can be simplified by setting E = e x+y u, then the function u satisfies 
the equation 



6 PAVEL GUSEV, VALENTINA KIRITCHENKO, AND VLADLEN TIMORIN 

and the boundary value conditions u(x, 0) = u(0,y) = 1. We can now look for 
solutions u that have the form v(xy), where v is some smooth function. This function 
must satisfy the initial condition v (0) = 1 and the ordinary differential equation 

tv"(t) + v\t) - v{t) = 0. 

It is known that the only analytic solution of this initial value problem is Io(2\/i), 
where Iq is the modified Bessel function of the first kind. Thus Io(2y/xy) is a 
partial solution of the boundary value problem u xy = u, u(x, 0) = u(0,y) = 1. 
The solution of this boundary value problem is unique (note that the boundary 
values are defined on characteristic curves!). Therefore, we must conclude that 
E(x,y) = e»»I (2 y /xy). 

The proof of Theorem 11.21 is very similar to the proof of Theorem 11.11 Let G* k be 
the sum of all terms in G k that are divisible by y\ . . . y k , i.e. 

Then, similarly to a formula obtained for E£, we have 

Gl = T, C ^~ Q1 ' ' ' yl' ak E A°°(x j )y j . 

Applying the operator A x . . . A*, to both sides of this equation, we obtain Theorem 
11.21 Similarly to the proof of Theorem 11.11 we need to use that 

A 1 ...A k (G* k ) = A 1 ...A k (G k ) 

and that 

A- . . . Af (G k ) = . . . yp* • ,X ^'' , 

We will now discuss several examples. 

Example: k — 1 and k — 2. For k — 1, we have the following equation: AiGi = 
Gi, i.e. Gx(yi) — C7i(0) = yiGi{y\). Knowing that Gi(0) = 1, this gives 

G 1 {y 1 ) = 1 + y x + y\ + ■ ■ ■ = 

l — 2/i 

Suppose that k = 2. Set G = G2, x = y±, y = y^. The function G satisfies the 
following equation 

A x A y G = A X G + A y G. 

Note that G(x, 0) = and G(0,y) = Gi(y). Therefore, the right-hand side can 

be rewritten as 

1-3/ _| l — x 

x y 

The left-hand side is 

' q 1 \ -, / ri J 1 



A, 




1— x I L I l—x l—y 
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Solving the linear equation on G thus obtained, we conclude that 

G 1 



1-x-y 



Example: k — 3. We set G = G 3 , x — y±, y — y 2 and z = y 3 . The function G 
satisfies the following equation: A x A y A z G = (A x + A y )(A y + A Z )G. This equation 
can be rewritten as follows: 

A 2 G = G(l-x-y-z)-l 

y xyz 

Suppose that G = G(x, 0, z) + A(x, z)y + . . . , where dots denote the terms divisible 
by y 2 . Then we have 

A 2 G — G — G(x, 0, z) - A(x, z)y = G - - A(x, z)y. 

Substituting this into the equation, we can solve the equation for G in terms of A: 

c _ -xz + y(l-x-z)(l-A(x,z)xz) ^ 
(l-x-z)(y-(x + y)(y + z)) 
Since the power series 1 — x — z is invertible, it follows that G has the form 

a + by 
V- (x + y)(y + zY 

where a and b are some power series in x and z. Let A and // be the two solutions 
of the equation y = (x + y)(y + z), namely, 



1 - x - z ± y/l - 2(x + z) + (x - z) 2 
2 • 

The signs are chosen so that, at the point x = z = 0, we have A = 1 and /i — 0. 
Then 

1 c d 

y - (x + y)(y + z)) y-X y - fi' 

where c and d are some power series in x and z. Note that, since (y — A)" 1 makes 
sense as a power series, c(a + by)/{y — A) can be represented as a power series in x, 
y and z. Thus the function d(a + by)/(y — /i) must also be representable as a power 
series in x, y and z. However, this is only possible if the numerator is a multiple of 
the denominator, i.e. (a + by) = e(y — fi), where the coefficient e is a power series 
of x and z. It follows that G is equal to e(y — A)" 1 . The coefficient e can be found 
from the condition G(x, 0, z) = 1 _l_ z - 

1 A 



G 



1 — x — zX — y 



2xz — y(l — x — z) — yyl — 2{x + z) + (x — z) 2 
2(1 - x - z)((x + y)(y + z)- y) 
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Figure 1. Triangular tables T s containing the numbers V k s m . South- 
west corners of these tables are located at (0, 0). 

4. Proof of Theorem 11.41 

In this section, we will prove Theorem 11.41 which expresses the numbers Vj~/ >m as 
coefficients of certain Laurent polynomials. The numbers V^ m satisfy the following 
recurrence relation: 

,171— 1 

+ V k 

provided that k, £, m > 0, and the following initial conditions: 

Set Vkm = Vk,8-k-m,m- Then we can write the following recurrence relations on 
the numbers V k s m : 

v? = Vu~} + Vu' 1 , + vrl i + Vu~ l 

* k,m Y k—l,m ' k,m—l ' k— l,m— 1 1 k,m 

provided that k ^ 1, m^l, k + m ^ s — 1, and 

k,m k—l,m ' k,m—l ' k— l,m— 1 

provided that k + m = s. 

For a fixed s, we can arrange the numbers V k s m into a triangular table T s of size 
s as shown on Figure [TJ Namely, the number V k s m is placed into the cell, whose 
southwest (lower left) corner is at position (k,m). The next table T s+1 can be 
obtained from the table T s as follows. First, we add to every element of T s its 
south, west and southwest neighbors. Next, we add a line of cells, whose positions 
(k, m) satisfy the equality k + m = s. In every cell of this line, we put the sum of 
the south and west neighbors. Note that, by construction, the boundary of every 
table T s consists of binomial coefficients. 

Consider the generating function G = G3 for the numbers Vk/, m - The splitting 
of G into homogeneous components can be obtained by expanding the function 
G(xy, y, zy) into powers of y. We set 

00 

G(xy, y, zy) = g s (x, z)y s 

s=0 
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Figure 2. The skew-symmetric tables T s . 
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Figure 3. The rules of generating the tables T s . 

Then we have 

s s—k 

Thus the coefficients of the polynomial g s are precisely elements of the table T s . 
The recurrence relations on the numbers V£ m displayed above imply the following 
property of the generating functions g s : 

Proposition 4.1. The polynomials g s satisfy the following recurrence relations: 

g s+1 = (1 + x + z)g s + r <s (xzg s ), 

where the truncation operator r<: s acts on a polynomial by removing all terms, whose 
degrees exceed s. 

Consider the polynomials 

h s (x, z) = g s (x, z) - (xzYgsiz' 1 , x' 1 ). 

Geometrically, these polynomials can be described as follows. Let T s denote the 
table, into which we put all coefficients of the polynomial h s , see Figure [2j The 
lower left triangle of size s — 1 is the same in the tables T s and T s . The table T s 
is skew-symmetric with respect to the main diagonal. These two properties give a 
unique characterization of the tables T s . 
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The rules, by which the tables T s are formed, are the following (see Figure |3]). 
The first table T 1 is by definition the left-most table shown on Figure EJ The 
next table T s+1 is obtained inductively from the preceding table T s in two steps. 
In the first step, we add to every element of T s its immediate west, south and 
southwest neighbors. In the second step, we modify elements in two diagonals of 
the table, namely, the elements, whose positions (mesured by southwest corners) 
(k,m) satisfy the equality k + m = s or k + m = s + 2. To the cell at position (k,m), 
where k + m = s, we add the binomial coefficient ( k ~^ n ) ■ From the cell at position 
(k + 1, m + 1), we subtract this binomial coefficient. 

We have the following recurrence relation on the polynomials h s : 

h s+1 = h s (l + x)(l + z) + (1 - xz){x + z) s , 

which does not contain truncation operators. Therefore, the generating function 
H = hsV s satisfies the following linear equation: 

H = 1 + y((l + x)(l + z)H + (1 - xz){l - y(x + z))- 1 ). 

Solving this equation, we find that 

H = y(i-xz) 

(l-y(x + z))(l-y(l + x)(l + z))' 

Knowing the generating function H, we can now obtain an explicit formula for 
the polynomials h s , namely, 

h s {x, z) = i— ^ ((1 + x) s (l + z) s -{x + z) s ) . 
1 + xz 

Theorem 11.41 is thus proved. 
Open problems. 

(1) Prove or disprove: the generating function G4 is algebraic. Note that G\ and 
G2 are rational, and G3 is algebraic. 

(2) Deduce differential or difference equations on the generating functions for 
the /-vectors and for the modified /i-vectors of Gelfand-Zetlin polytopes. 
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